This invention relates generally to human/computer interfaces, and more particularly to human/computer interfaces with force feedback that can operate over a network.
The Internet has, of late, become extremely popular. While the use of the Internet has been prevalent for many years now, its use has been limited by the arcane and difficult commands required to access the various computers on the network. To address this problem, a protocol known as the xe2x80x9cWorld Wide Webxe2x80x9d or xe2x80x9cWWWxe2x80x9d was developed to provide an easier and user-friendlier interface for the Internet. With the World WidE Web, an entity having a domain name creates a xe2x80x9cweb pagexe2x80x9d or xe2x80x9cpagexe2x80x9d which can provide information and, to a limited degree, some interactivity.
A computer user can xe2x80x9cbrowsexe2x80x9d, i.e. navigate around, the WWW by utilizing a suitable web browser and a network gateway (e.g., an Internet Service Provider (ISP)). Currently, popular web browsers, include Netscape(copyright) Navigator(copyright) made by Netscape Corporation of Mountain View, Calif., and Internet Explorer made by Microsoft(copyright) Corporation of Redmond, Wash. A web browser allows a user to specify or search for a web page on the WWW, and then retrieves and displays web pages on the user""s computer screen.
The Internet is based upon a transmission protocol known as xe2x80x9cTransmission Control Protocol/Internet Protocolxe2x80x9d (or xe2x80x9cTCP/IPxe2x80x9d for short), which sends xe2x80x9cpacketsxe2x80x9d of data between a host machine, e.g. a server computer on the Internet, and a client machine, e.g. a user""s personal computer connected to the Internet. The WWW is an Internet interface protocol which is supported by the same TCP/IP transmission protocol. Intranets are private networks based upon Internet standards; since they adhere to Internet standards, can often use the same web browser software and web server software as are used on the Internet.
A web page typically includes static images, animated images (e.g. video), and/or text. The images and text are specified in a xe2x80x9cHyperText Mark-up Languagexe2x80x9d (xe2x80x9cHTMLxe2x80x9d) file that is sent from the web server to the client machine. This HTML file is parsed by the web browser in order to display the text and images on the display of the client machine. Other standardized languages or protocols are also being developed for use with the Internet and the World Wide Web. For example, the Virtual Reality Modeling Language (VRML) is used to provide visual virtual 3-D environments and allow one or many users to navigate through and interact as xe2x80x9cavatarsxe2x80x9d in such an environment using a web browser or other software on a client computer system.
Furthermore, additional functionality may be provided in web pages with information downloaded over the WWW in the form of scripts or programs. Scripting, typically in the form of VBScript or JavaScript, allows a series of instructions to be perfonned on the client computer once the instructions have been downloaded in a web page. Programs can be provided in such standard languages as Visual Basic, C++, or currently in the form of Java xe2x80x9cappletsxe2x80x9d or ActiveX(copyright) controls. Java includes a platform-independent interpreter running on the client machine that executes downloaded applets within web pages or other programs, e.g., to display animated images, retrieve data over the WWW, output feedback to the user, or perform operating system tasks. ActiveX controls similarly execute on the client computer once the program instructions are resident on the client. ActiveX controls are programmable objects that can be embedded into Web pages and may be written in any (platform-specific) language. Java and ActiveX controls can add functionality to a Web page that would normally be difficult, or even impossible, using HTML or scripting languages. ActiveX controls can also be controlled with a scripting language. Alternatively, functionality can be added to a web page through the use of xe2x80x9cplug-insxe2x80x9d, which are application programs running in conjunction with certain web browsers to parse plug-in-specific code in the web page which the browser cannot understand.
Other WWW-related functionality includes Dynamic HTML. Dynamic HTML is a set of features currently incorporated in browsers such as Microsoft Internet Explorer that enable authors to dynamically change the rendering and content of an HTML document. Using Dynamic HTML, a content developer or programmer can access the attributes of a document""s contents or objects (such as an object""s position on the page and type). In addition, event messages are generated when a user interacts with the web page content (such as when a user clicks on a graphical button image). The features of Dynamic HTML can be elicited through the use of VBScript or JavaScript scripts embedded in a Web page or programmatically through Visual Basic or C++.
The Internet and the WWW also permit sound data to be transmitted over the Internet. For example, references to sound files can be embedded in HTML pages and can be played by the web browser. Data xe2x80x9cpacketsxe2x80x9d coded in TCP/IP format can also be sent from one client machine to another over the Internet to transmit sound data. This last-mentioned technique forms the basis for Internet telephony.
While the transmission of visual images (both static and dynamic), text, and sound over the Internet is well-known, the transmission of other types of sensory data has not been well explored. In particular, the transmission of data over the Internet pertaining to the sense of touch and/or force has not been established. xe2x80x9cForce feedbackxe2x80x9d allows a user to experience or xe2x80x9cfeelxe2x80x9d tactile sensations as provided through computational information. Using computer-controlled actuators and sensors on a force feedback device, a variety of realistic sensations can be modeled and experienced by the user. This useful and highly immersive sensory modality for interacting with the Internet has hereto been unavailable.
In addition, there needs to be tools to allow web page authors to quickly and easily add forces to web page content or adjust/modify existing forces as desired. Such tools should preferably allow an author or user to intuitively include forces in web pages without needing a knowledge of force feedback instructions or programming constructs. In addition, the author should be provided with an intuitive way to design or adjust the forces. Thus, a tool is needed for assisting the user, programmer or developer in intuitively and easily creating web pages and in setting force feedback characteristics to provide desired force sensations in web pages.
The present invention is related to the transmission/reception of information pertaining to force feedback to provide feel in transmitted information such as web pages. The force feedback provided by the methods and apparatus of the present invention enhance the sensory experience of the user to provide a richer, more interesting, and more enjoyable interaction with information received over networks.
In one aspect of the present invention for providing force feedback over a network, information is received by a client machine over a network from a server machine. The client machine (typically a personal computer) has a visual display and a force feedback interface device. Force feedback interface devices can include force feedback mice, joysticks, finger cradles, trackballs, steering wheels, and yokes. The force feedback interface device includes sensors and actuators and preferably a local microprocessor for communicating with the client machine and controlling the actuators of the interface device.
Several embodiments are disclosed in which forces are implemented based on the information received over the network. The disclosed embodiments emphasize the World Wide Web or the Internet as the network provided between client and server computers, where web page information received by the client is in HTML or other standard format and a web browser running on the client machine displays the web page. One embodiment provides authored force effects, which are effects particular to a web page that are customized for each web page object and are downloaded with the web page. The received web page information includes screen display information representing a visual layout of a web page and force feedback information related to providing a feel sensation. Input information from the interface device is used to position a cursor with respect to the visual layout of the web page. A force feedback signal is provided based upon the input information and the web page information, and the received force feedback information includes a call to a force feedback program running on the client machine that provides the force feedback signal. The force feedback signal is received by the interface device, which outputs computer-controlled physical force feedback to the user correlated with the visual layout of the web page. The force feedback program running on the client can be ActiveX control, a Java applet, a plug-in, etc. In one preferred embodiment, the ActiveX control is called by script instructions, e.g. in JavaScript, included in the received force feedback information. The script instructions can provide downloaded force effect parameters to the force feedback program to output a desired force effect.
In a different embodiment, generic force effects are implemented. Generic effects are applied uniformly to all web page objects of a particular type. A web page need not include force information to allow generic force effects to be applied to the web page content. Thus, a received web page includes screen display information representing web page objects. The client determines which web page objects are force web page objects to be associated with at least one generic force effect, where the force web page objects are of a predefined type. A generic force effect is assigned to each type of web page object as defined by effect information derived from the client machine. Generic force effects are output when a user-controlled cursor interacts with a force web page object. A force control program running on the client machine can detect whether the cursor is contacting one of the force web page objects and output the appropriate force feedback signals to the interface device. In another embodiment, a downloaded web page can be preprocessed before being displayed to add force feedback information to the web page. The client can perform the preprocessing, or a proxy server can preprocess the web page before sending the web page to the client. In other embodiments, both generic effects and authored effects can be provided for a particular web page. If both types of force effects are used, either type of force effect can be overridden by the other depending on user preferences.
In another aspect of the present invention, a web page authoring interface is provided that includes the ability to add force sensations to a web page. The web page authoring interface is displayed on a display device of a computer and displays a web page including web page objects. Input from a user is received to the authoring interface and selects a web page object and a force effect to be associated with the selected web page object. A web page is output by the interface, including the web page objects and including force information to allow the force effect to be implemented when the web page is displayed by a client machine after being received from a server machine over a network. The authoring tool can include a force design interface for creating or modifying the force effect, or a separate force design interface can accessed. The force effects added to web page objects can also be experienced directly by the author during the design process. In one embodiment, the author is also able to spatially designate an area of a web page object to be associated with a selected force effect, e.g. a graphical mark or outline can designate sub-areas of an image or text to be associated with forces. In addition, the user can preferably associate sound data with a web page object, such that when said force effect is output, a sound is output to the user synchronized with the force effect. Finally, a predefined graphical identifier can be inserted into the web page to indicate to a user that the web page provides force effects, and which may also be a link that causes a linked force feedback resource web page to be downloaded.
The present invention provides several ways to implement force feedback over a network such as the World Wide Web. Embodiments included herein allow standard web pages without any force content to be assigned forces on a client machine including force feedback capability. In other embodiments, a web page author can include specific force effects in a web page to any desired level of customization. The described force feedback web page authoring tool allows force effects to be easily and quickly included in web pages to foster greater diversity and widespread use of feel in web pages.
These and other advantages of the present invention will become apparent upon reading the following detailed descriptions and studying the various figures of the drawings.